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Abstract. For continuous-time Markov chains, the model-checking problem with respect 
to continuous-time stochastic logic (CSL) has been introduced and shown to be decidable 
by Aziz, Sanwal, Singhal and Brayton in 1996 [HE]. Their proof can be turned into an ap- 
proximation algorithm with worse than exponential complexity. In 2000, Baier, Haverkort, 
Hermanns and Katoen [3] [5] presented an efficient polynomial-time approximation algo- 
rithm for the sublogic in which only binary until is allowed. In this paper, we propose such 
an efficient polynomial-time approximation algorithm for full CSL. 

The key to our method is the notion of stratified CTMCs with respect to the CSL prop- 
erty to be checked. On a stratified CTMC, the probability to satisfy a CSL path formula 
can be approximated by a transient analysis in polynomial time (using uniformization). 
We present a measure-preserving, linear-time and -space transformation of any CTMC into 
an equivalent, stratified one. This makes the present work the centerpiece of a broadly 
applicable full CSL model checker. 

Recently, the decision algorithm by Aziz et al. was shown to work only for stratified 
CTMCs. As an additional contribution, our measure-preserving transformation can be 
used to ensure the decidability for general CTMCs. 



Continuous-time Markov chains (CTMC) play an important role in performance evaluation 
of networked, distributed, and biological systems. The concept of formal verification for 
CTMCs was introduced by Aziz, Sanwal, Singhal and Brayton in 1996 [HL?]- Their seminal 
paper defined continuous-time stochastic logic (CSL) to specify properties over CTMCs. 
It showed that the model checking problem for CTMCs, which asks whether the CTMC 
satisfies a given CSL property, is decidable, using algebraic and transcendental number 
theory. Their proof is constructive, so it can be turned into an approximation procedure 
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for the relevant probabilities. However, its complexity may be worse than exponential in 
the size of the formula. 

The characteristic construct of CSL is a probabilistic formula of the form "P <p (</?), 
where p £ [0,1]. Here (p is a path formula; more concretely, it is a multiple until formula 
fi Ui x fi Ui 2 . . . Ui k l fk where k > 2. The formula V <p (ip) expresses a constraint 
on the probability to reach an /^-state by passing only through (zero or more) /i-, /2-, 
. . . , //j_i-states in the given order (together with a timing constraint indicated by the 
intervals I±, . . . , Ik-i)- The key to solve the model checking problem is to approximate this 
probability Pr s (ip) closely enough to decide whether it is < p. The decision procedure in [2] 
first decomposes the formula into (up to) (k — l) fc ~ x many subformulas with suitable timing 
constraints. For each subformula, it then exploits properties of algebraic and transcendental 
numbers, but the corresponding algorithm is unfortunately impractical. In 2000, Baier et 
al. [U [5] presented an approximate model checking algorithm for the case k = 2. This 
algorithm is based on transient probability analysis for CTMCs. More precisely, it was 
shown that Pr s (y>) can be approximated, up to an a priori given precision e, by a sum of 
transient probabilities in the CTMCs. Their algorithm then led to further development 
of approximation algorithms for infinite CTMCs 112] and abstraction techniques [15] . 
More importantly, several tools support approximate model checking, including PRISM [17] 
and MRMC [16]. 

Effective model checking of full CSL with multiple until formulas (k > 2) is an open 
problem. This problem is gaining importance e. g. in the field of system biology, where one is 
interested in oscillatory behavior of CTMCs [U[T9]. More precisely, if one intends to quantify 
the probability mass oscillating between high, medium and low concentrations (or numbers) 
of some species, a formula like Vyo^ihigh XJ\ X medium Ui 2 low Ui 3 medium Uj 4 high) is 
needed, but this is not at hand with the current state of the art. In CTL, multiple until 
formulas like \/(high U medium U low U medium U high) do not increase expressivity 
because they are equivalent to something like \/(high U \/{medium UM{. . . U high))). 

In this paper we propose an approximate algorithm for checking CSL with multiple 
until formulas. We introduce a subclass of stratified CTMCs, on which the approximation 
of Pr s (<^) can be obtained by efficient transient analysis. Briefly, a CTMC is stratified 
with respect to <p = f\ Ui 1 fi Ui 2 ■ ■ ■ fk, if the transitions of the CTMC respect the order 
given by the fi. This specific order makes it possible to express Pr s (c^) recursively: more 
precisely, it is the product of a transient vector and Pr s / (<//), where (p' is a kind of suffix 
subformula of ip. Stratified CTMCs are the key element for our analysis: in a stratified 
CTMC, the problem reduces to a transient analysis, for which efficient implementations 
using uniformization |10] exist. Thus, we extend the well-known result [5] for the case of 
binary until to multiple until formulas. 

For a general CTMC, we present a measure-preserving transformation to a stratified 
CTMC. Our reduction is described using a deterministic finite automaton (DFA) over the 
alphabet 2^ fl '-' fk \ The DFA accepts the finite word w = W1W2 • • • w n if and only if the 
corresponding set of time-abstract paths in the CTMC contributes to Pv s (ip), i. e., it respects 
the order of the fi. The transformation does not require to construct the full DFA, but only 
the product of the CTMC and the DFA. We show that the product is a stratified CTMC, 
and moreover, the measure Pr <,((/?) is preserved. This product can be constructed in linear 
time and space in the size of the CTMC and k. Thus our method will be useful as the 
centerpiece of a full CSL model checker equipped with multiple until formulas. 
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Recently, the decision algorithm by Aziz et al. was shown to produce erroneous results 
on some non-stratified CTMCs [13]. Still, their algorithm is correct on stratified CTMCs. 
As an additional contribution, our measure-preservation theorem ensures the decidability 
of CSL model checking for general CTMCs. 

Overview of the article. Section [2] sets the ground for the paper. In Section [3] we introduce 
stratified CTMCs formally. The first main result is shown in Section 2J it constructs a 
DFA for an until formula, and then shows that the product is a stratified CTMC and 
the relevant measures are preserved. Section [5] discusses the computations in the product 
CTMC. A model checking algorithm is presented in Section [6] Section [7] discusses related 
work, and the paper is concluded in Section El 



This section presents the definition of Markov chains, probability space, transient and 
steady-state distributions. For details please refer to [20 ], [T8 ] [5]. 

2.1. Markov Chains. 

Definition 2.1. A labeled discrete-time Markov chain (DTMC) is a tuple V = (5, P,L), 
where S is a finite set of states, P : S x S — > [0, 1] is a probability matrix satisfying 
Yls'es -f > ( s ' s ') e 1} f° r an s £ S, and L : S — > 2 AP is a labeling function. 

A labeled continuous-time Markov chain (CTMC) is a tuple C = (5, R, L), where S 
and L are defined as for DTMCs, and RiS'xS'-y ]R>o is a rate matrix. 

For A C S, define R(s, A) := XL'eA R-( s > s ')> an d let E(s) := H(s,S) denote the exit 
rate of s. A state s is called absorbing if E(s) = 0. If R(s, s') > 0, we say that there is a 
transition from s to s' . 

The transition probabilities in a CTMC are exponentially distributed over time. If s is 
the current state of the CTMC, the probability that some transition will be triggered within 
time t is 1 — e~ E ^ 1 . Furthermore, if R(s, s') > for more than one state s' , the probability 
to take a particular transition to s' is - • (l — e~ E ^ t ). The labeling function L assigns 
to each state s the set of atomic propositions L(s) C AP which are valid in s. 

A CTMC C (and also a DTMC) is usually equipped with an initial state Sj n it £ S or, 
more generally, an initial distribution : S — > [0, 1] satisfying Ssg5 a mit(s) = 1. 

Paths and probabilistic measures. A (sample) path is a right-continuous function a : M>o — > 
S (with the discrete topology on S). Then, a(t) denotes the state occupied at time t. 

For i 6 N, let as[i] = Sj denote the (i + l)-th state visited, and ax[i] = U denote the 
time spent in o~s[i\- For finite paths, <tt[^] is defined to be oo if o~s[n] is the last (absorbing) 
state. Let Path c denote the set of all (finite and infinite) paths, and Path c (s) denote the 
subset of those paths starting from s. 

We sometimes use a different notation to describe a path, namely a finite sequence 
a = so*oSi*i ■ ■ ■ &n (meaning that as[i] = *i and ot[«] = U for all i < n, and as[n] = s n 
is an absorbing state), or an infinite sequence a = so*o s i*i ■ ■ • if no absorbing state is 
hit. The relation between the two notations is: a(t) = Sj where i is the smallest index 
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with t < X^=oA? ( as remarked by [T8j p. 170], we have to use a strict inequality here for 
technical reasons, not the non-strict inequality as in [5].). 

Let so, si, . . . , Sk be states in S with R(sj, Sj+i) > for all < i < k. Let Iq, I\, . . . , Ik-i 
be nonempty intervals in M>o- The cylinder set Cyl(so, Io, . . . , Sfc_i, 4-1, Sfc) is defined by: 

Cyl(so, Iq, . . . , Sk-i, Ik-i, Sk) '■= {o~ G Path c | V0 < i < k. as[i] = SjAVO < i < k. G 

Let F{Path c ) denote the smallest u-algebra on Path c containing all cylinder sets. For 
initial distribution a : S — >• [0, 1], a probability measure (denoted Pr^) on this cr-algebra is 
introduced as follows: Pr^ is the unique measure that satisfies: Pr^(C?/Z(s)) equals a(s), 
and for k > 0, 

Pv c a (Cyl(s , 1 , ••• , 4-i, s fc )) = P^(C^( So , Io, • • • , 4-2, Sfe-i)) ■ • ^(4-i) 

where 7j(4-i) := exp(— £?(s^_i) inf 4-i) — ex P( — E(sk-i) sup 4-i) is the probability to 
take a transition during time interval 4-1- (As a consequence, the probability of a cylinder 
set containing a point interval [£, t] is 0.) If a(s) = 1 for some state s S S, we sometimes 
simply write Prf instead of Pr^. We omit the superscript C if it is clear from the context. 

Transient and steady-state probability. Starting with distribution a, the transient probabil- 
ity vector at time t, denoted by n(a,t), is the probability distribution over states at time 
t. If t = 0, we have 7r(a,0)(s') = a(s'). For t > 0, the transient probability is given by: 
ir(a, t) = tt (a, 0)e^ 4 where Q := R— Diag(E) is the infinitesimal generator matrix. Diag(E) 
denotes the diagonal matrix with Diag(E)(s, s) = E(s). The steady-state distribution is 
defined as the limit lim^oo iv(a, t), which always exists for finite CTMCs. 

2.2. Deterministic Finite Automata. 

Definition 2.2. A deterministic finite automaton is a tuple B = (T,,Q,qi n ,6,F), where £ 
is a finite alphabet, Q is a finite set of states, qi n € Q is an initial state, 5 : Q x Y, Q is & 
partial transition function, and F C Q is a set of final states. 

We call a finite sequence w = W1W2 ■ ■ ■ w n over £ a word over £. iu induces at most 
one path a(w) = qoq± . . . q n in B where qo = qi n and qi = 5{qi-\,Wi) for i = 1, . . . ,n. This 
word w, and also the corresponding path o~(w), is accepting if <r(iu) exists and q n G F 1 . 

2.3. Continuous Stochastic Logic (CSL). We consider the branching-time temporal 
logic Continuous Stochastic Logic (CSL) introduced by Aziz et al. [2], which allows us to 
specify properties over CTMCs. Its syntax is defined as follows: 

$ : = a I -1$ I $ A $ I V< p ((f) 

if := $1 $ 2 E4 2 • • • U h _ x <S> k 

where a G AP is an atomic proposition, I\,Ii,... C IR> are nonempty left-closed intervals 
with rational bounds, < G {<, <, >, >}, p G Q fl [0, 1], and k > 2. We use the abbreviation 
0/$ = (-■(a A -■a)) <3?, for an arbitrary atomic proposition a. The syntax of CSL 
consists of state formulas and path formulas: we use $1, ^f\, . . . for state formulas and 
(f, <pi,ip, ipi, ■ ■ ■ for path formulas. 
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Let C = (S, R, L) be a CTMC with s £ S. The semantics of most CSL state formulas 
is standard: s \= a iff a G L(s); s |= iff s ^ <£; s |= $ A ^ iff s |= $ and s |= ^. For 
probabilistic formulas, we have: 

s |= V< p ((p) iff Pr s {o" € Pa£/i | <r |= p) < p 

where Pr s {cr 6 Pai/i | a |= y}, or Pr s (</?) for short, denotes the probability measure of the 
set of all paths which start with s and satisfy (p. 

The satisfaction relation for CSL path formulas is defined as follows: let a be a path, 
and let p> = <l?i Ui 1 $2 Uj 2 . . . be a path formula. Then a |= (p if and only if there 
exist real numbers < t\ < t 2 < . . . < tk~\ such that a{tk-i) |= and for each integer 
< i < k we have (ti £ Ii) A (Vt' £ [tj-i, U)){a{t') |= <£j), where to is defined to be for 
notational convenience. 

For a CSL path formula p = $1 f/[ai,6i) $2 ^[a 2 ,&2) ^3 w ith 02 < a±, one can replace 
the second interval by [01,62) without changing the set of paths that satisfy the formula. 
Thus, we shall assume that the left endpoints - and similarly, the right endpoints - of the 
intervals in multiple until formulas are always nondecreasing. 

3. Stratified CTMCs 

The main challenge of model checking is the computation and the approximation of the 
probability Pr s (p). We now introduce the class of stratified CTMCs. This is the key 
for the computation of Pr s [p). For now, the path formula <p> contains pairwise different 
atomic propositions as subformulas. In Section \6. 11 we shall see that this definition is easily 
generalized to formulas containing more complex subformulas. 

Let C = (S, R, V) be a CTMC. Let p = f\ Ui l f 2 Uj 2 ... be a CSL path formula 
with pairwise different atomic propositions. Moreover, we let F := /2, . . . , fk}, and C 
be an order on F such that C fj iff i < j. For a state s, if the set L(s) n F is not empty, 
we let f^ in := mine L(s) n F denote the least element fi with respect to the order C. If 
such fj does not exist, we define / min := _L. 

Definition 3.1 (Stratified CTMC). We say that C is stratified with respect to p iff for all 
s±, S2, it holds that: 

. If = 1 or = f k , then R( 5l , s 2 ) = 0. 

. Otherwise (i. e., + J_ and + f k ), if R( Sl , s 2 ) > and fX + then C 

A state s with /^ in = _L is a bad state, and a state with /^ in = fk is a good state. 
(Note that there may be other states satisfying fk as well.) Both good and bad states are 
absorbing. The intuition behind Def. 13.11 is that paths reaching bad states will not satisfy 
<p, while those reaching good states or other /^-states may satisfy p> (provided the timing 
constraints are also satisfied). 

Example 3.2. Consider the path formula p := f x U[ 0j2 ) js- ^[2,4) fz ^[2,4) h ^[3,5) h- 
The CTMC in Fig. CD is not stratified with respect to p: we have R(s2,.si) > 0, however, 
/mm = A 2 /1 = /min- Deleting this edge and the transition out of S4 would result in a 
stratified CTMC with respect to p. □ 
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Figure 1: A non-stratified CTMC. 



The notion of stratified CMTCs is the key to an efficient approximation algorithm. 
The essential idea is that we can reduce the model checking problem to one on a similar, 
stratified CTMC that preserves the relevant reachability probabilities. Further, our notion 
of stratified CTMCs solves a semantical problem in [2]: please refer to Section [631 for details. 



4. Product CTMC 

Given a CTMC and a CSL path formula <p, in this section we construct a stratified CTMC 
with respect to tp preserving the probability to satisfy tp. We first construct a deterministic 
finite automaton for <p> in Subsection l4.1i Then, in Subsection HT2] we build a product CTMC 
with the desired property. 

4.1. Automaton for a CSL Formula. For a path formula <p = fi f 2 Uj 2 . . . we 

first construct a simple deterministic finite automaton (DFA) that describes the required 
order of ft-, f 2 -, /^-states. 

Definition 4.1 (Formula automaton). Let <p = fi Ui 1 f 2 Uj 2 ■ ■ ■ fk be a CSL path for- 
mula with pairwise different atomic propositions. Then, the formula automaton B v = 
(E,Q,q in ,8,F) is defined by: £ = 2^ fl '-' / ^, Q = {qi,q 2 , ■ ■ .,qk-i,qk, -L} with q in = q 1 and 
F = {ill ■ ■ ■ -i Qk}- For a G S, the transition relation 5 is defined as follows: 

(1) %j, a) = qj if % < k; i < j; fi, f i+1 . . . fj-i a; and fj G a; 

(2) S(qi, a) = J- if i < k and the above clause does not apply; 

(3) _L and qk are absorbing. 

As states _L and qk have no outgoing transitions, S is a partial transition function. Thus 
formula automata are actually partial DFAs. The words accepted by B v are finite traces 
w G X* that can be extended to a trace ww' G S w that satisfies the time-abstract formula 
of the form f\ U f 2 U . . . U fk- The constructed finite automaton for this special class 
of formulas is deterministic, the number of states is linear in k. The number of transitions 
is (k — l)2 fc ; however, as we will see later, the product can be constructed in time (and size) 
linear in the size of the CTMC and in k. 

Example 4.2. In Fig. [2]the formula automaton for k = 4 is illustrated. The initial state is 
qi, final states are marked with a double circle. The transition labels indicate which subsets 
of AP are acceptable. For example, we have S(q±, {/i}) = 8(qi, /2D = qi, as both sets 
satisfy f\. □ 
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Figure 2: B v for tp = fx U f 2 U / 3 U / 4 



4.2. Product CTMC. 

Definition 4.3 (Product CTMC). Let C = (S, R, L) be a CTMC and (p = fi U h f 2 U h 
. . . ff~ a path formula with pairwise different atomic propositions. Let B v be as constructed 
above. The product C x B v is a CTMC (S",R',L') where: 

(1) 5' = 5 x Q, 

(2) R'((s,ft), (V,g')) equals R(s, sf) if s |= C /i V f i+1 V ■ ■ ■ V and g' = S{q i ,L(g r ) n 
{/li • • • i /fe})j and equals otherwise, 

(3) the labeling function is defined by: 

• L'(s, qi) = L(s) n {fi, f i+ i, . . . , f k } for 1 < i < k, 

• L'(s,±) = 0. 

(4) Given an initial distribution a : S — > [0, 1] of C, the initial distribution of the product 
a' : S x Q — > [0, 1] is defined by: a'(s,g) equals q(s) if q = 5(qi n ,L(s) D {/i, . . . , 

and equals otherwise. 

The product CTMC contains two kinds of absorbing states. In general, states (s, q) with 
s Vi=i /« are absorbing in the product, as well as states reached through a transition that 
does not follow the prescribed order of fi. These two kinds of states can be considered bad 
states. On the other hand, good states of the form (s, g&) with s \= fk are also absorbing. 
The behavior after such an absorbing state is irrelevant for the probability to satisfy (p. 

Example 4.4. Consider the CTMC in Fig. [lj and consider the path formula ip\ := fi £/[o,2) 
h ^[0,2) h ^[0,2) fi U [Q,2) fa- Tne Path a i '■= s Q s 1 s 3 S2S 4: . . . does, if s 4 is reached before 
time 2, satisfy (p±; however, the path 02 '■= SQS1S2S1S3S2S4 . . . does not. The product of this 
CTMC with B v>1 is the CTMC depicted on the left of Fig. [3j which is stratified with respect 
to ipi. State (54, q§) is a good state - paths reaching this state before time 2 correspond to 
paths satisfying ip\ in Fig. [T]-, while (S3, _L) is a bad state. 

For the same CTMC in Fig. [TJ consider the path formula <^ 2 := fx ^[1,3) /2 ^[1,3) 
/ 3 ^[1,3) fi- The product CTMC C x B V2 is depicted on the right of Fig. El This product 
is stratified with respect to if2- The absorbing state (^2,54) is a good state. □ 
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/l >/2 >/4 fa fa fl,f2,fa 

Figure 3: The reachable part of the product CTMC C x B !fl (left) and C x Bp 2 (right). 



For a CTMC C = (5,R,L) and a state s E S, we use C| s = (5',R',L') to denote the 
sub-CTMC reachable from s, i.e., S' C 5 is the states reachable from s, R' and L' are 
functions restricted to S' x S' and 5', respectively. 

Theorem 4.5 (Measure-preservation theorem). Let C = (S, R, L) 6e a CTMC and (p = 
fi Uj-y f2 Ui 2 ■ ■ ■ fk a path formula. Let Bp denote the formula automaton. For s 6 S, let 
s B = {s,8(q in ,L(s) n {fa, . . . ,/fc})). T/ien; 

(1) CxS,| Ss is stratified with respect to (p; 

(2) Pr c s (ip) = Ptf^'Bfr) = Pr^fa). 

Proof. We prove first that CxBp\ SB is stratified with respect to <p. Consider a state (s, q). By 

definition of the product CTMC, if (s, q) ty=cxB v Vi=i /*> tnen s V=c Vi=i /« or g G _L}, 
so state (s, g) is absorbing and therefore trivially satisfies the stratification conditions. Now 
assume that (s,q) |=cxB v Vi=i /*> 1 & -*-}> ano - moreover assume {s',q') is a state with 
R'((s, 5), (s',q')) > (with R' as in Def. 14.30 . By the definition of the transitions of Bp, 

(s 1 a') 

we have q = S(q,L(s') n {/1, . . . , fk})- Now assume 7^ _L: it remains to be shown 

that C / m 3 m <? ' ) - Let 1 < x < /c such that / x = /^ } , and let 1 < y < k be such that 

q = q y - The indices a/ and y' are defined similarly for (s',q'). By definition of transitions 
of Bp and product CTMC, it is routine to verify that x = y and x 1 = y'. Moreover, in 
Bp, q' = 5(q, L(s') n . . . , fk}) implies that y' > y, which shows that x < x' , proving 

As,q) |- As',q') 
J min — J min 

Now we prove the second clause. Obviously, states not reachable from sb can be safely 
removed, thus Pr^^s (<p) = Pi^ B ^(<p). We next prove that Pj^{<p) = Prf* 6 ^) by 
showing that a 1— > o~b (the canonical mapping from paths in C to paths in C x Bp) preserves 
the standard probability measures between the probability spaces. To this end, it is enough 
to show that given a cylinder set Cb over C x Bp, its reverse image C = {o-\cfb £ Cb} 
satisfies Pr^(C) = Pv c s * B *(C B ). 

Stated briefly, we now show that paths in C and in C x Bp correspond to each other 
because we only add some (bounded) information about the past to the states. 

Let us first describe the canonical mapping a 1— >• o~b- Assume given a path a = 
sotoS\ti ... in C. The corresponding path in C x Bp is as = {sq, q )to(s\, q l )t\ . . ., where 
q° = 5{q in , L(s Q ) H {fx, . . . , fk}) and q l+1 = 5(q l , L(s i+1 ) n {/1, . . . , /*.}) for all i > 1, as 
long as the {si,q l ) are not absorbing. However, if (s n ,q n ) is absorbing for some n, then 
ob is defined to be the finite path {so, q°)to(si, q x )t\ . . . (s n ,q n ), where {s n ,q n ) is the first 
absorbing state encountered. Note that a \=c 9? iff o~b |=CxB v V- 
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Let Cb = Cyl((so,q°), Iq, (si,q ' ), . . . , (s n ,q n )) and C be as above. By definition of a 
cylinder set, R'((sj, q l ), (sj+i, > for all i < n, therefore {si,q l ) is not absorbing 

(for i < n) and q t+1 = 5{q l , L(sj+i) fl {/i, . . . , /&}). Now assume that some path a = 
SytoS^tx . . . G C; then it must hold that s' = sq, to G Iq, s' x = si, t\ G ii, and 
= s n . Therefore, C C Cyl(so, Iq, s%, ■ ■ ■ , s n ). On the other hand, for all paths a G 
Cyl(so, Iq, si, ... , s n ), it is easy to prove that cfb G Cb- So, C 5 Cyl(so, Iq, si, . . . , s n ), and 
together, C = Cyl(sQ, Iq,s±,..., s n ). It is now an easy calculation to verify that Pr^(C) = 
Pr£ B *(C fl ). 

The reverse image of the set of C x ,6^-paths satisfying p is exactly the set of C-paths 
satisfying p. Since these sets are measurable, both can be decomposed into countable unions 
of corresponding cylinder sets in C and C x B v , respectively. Thus, the theorem follows. □ 

5. Characterizing the Probability Pr a ,(^) 

For a path formula p, together with a stratified CTMC with respect to tp, this section aims 
at a recursive characterization of the probability Pr a (p) starting from an arbitrary initial 
distribution a. 

We first introduce some notation. For an interval / and < x, we let / x denote the 
set {t - x | t G I A t > x}. For example, [3, 8) 5 = [0, 3). Then, for tp = f x U h f 2 U h ... f k 
and x < sup/i, we let <p x denote the formula f\ Ui^q x f 2 Ui 2 q x ■ ■ ■ fk- For 1 < < k, 
define fj^.j := \J J i=j , ff, for 1 < j < k, define tpj := fj Ui j f j+1 U Ij+1 . . . f k . As a degenerate 
case of Pi s (<pj), let Pr s (/j.) := 1 if s |= fk and otherwise. For $, we denote by C[3>] the 
CTMC obtained by C by making states satisfying $ absorbing - by cutting transitions out of 
all states satisfying <F Moreover, let 1$ denote the indicator matrix defined by: I$(s, s) = 1 
if s \= <3?, and I$(s, s') = otherwise. 

5.1. Left-Closed Intervals. For the moment, we restrict our attention to until formulas 
where all timing constraints have the form l{ = [cjj, hi). The following theorem characterizes 
the probability for this case: 

Theorem 5.1. Let p = f\ Uj 1 f 2 Uj 2 ■ ■ ■ fk be a CSL path formula with pairwise different 
atomic propositions, and assume all I{ = [a^&j) are left-closed. Let C = (S, R, L) be a 
stratified CTMC with respect to p. We write the vector (Pr^(^)) s£ 5 as Pr?\(^). 

(1) Assume < a\. Then, 

Pr c a (p)=n c ^(a,a 1 )-I fl -Fr c { . ) (pea 1 ) (5.1) 

where ir c ^^(a,ai) is the transient distribution at time a\ in the CTMC C[-i/i]. 

(2) Assume = a\ = . . . = dj-i < dj < b\ for some j G {2, . . . , k — 1}. Then, 

Pr c a {p) = 7T C ^-](a )% -) • I fl ^ • Prfj^ea,-) (5.2) 

(3) Assume = a± = . . . = aj-i < b\ < dj for some j G {2, . . . , k — 1}. Let j' < j be the 
largest integer such that b\ Ij'-i- Then, 

Pr c M = 7r c ^-](a, by) ■ l fjl . • Pr^'- fe - l] (^ © h) (5.3) 
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(4) Assume = ai = ... = dk-i- Let f < k be the largest integer such that b\ Ij'-i- 
Then, 

Pr ^) = 7r c ^(a, h) ■ I fj , h • Pr^- fc - J (^ h) (5.4) 

If bi = oo, we replace iT C ^ k \a,bi) in this equation by the corresponding steady-state 
distribution. 

The key idea of the theorem is a property- driven transient analysis. In the first clause 
we have a\ > 0, thus for any path a satisfying ip it must hold a(t) |= f\ for all t G 
[0,ai). Thus, we make all states satisfying -i/i absorbing, and compute the transient 
distribution Furthermore, the multiplication with the matrix 1^ removes 

the probabilities in states satisfying -i/i - thus resulting in a subdistribution. Starting 
with this subdistribution, the formula will also be reduced by duration a\. In the other 
clauses, we consider the interval [0, aj) or [0, which is the common prefix of the intervals 
I\, . . . , Ij—i- Thus during this time the formula must be satisfied. Here the assumption 
of stratification is crucial: otherwise one might be able jump forward and back between 
states satisfying f\ and fj, which is illustrated in the following example. 




fi f2 fi f2 h 



Figure 4: A CTMC with Pr So (/i U m f 2 U [Qtl) / 3 ) = 0. 

Example 5.2. Consider the CTMC depicted in Fig. H] and consider the path formula 
(p = fi C^[o,i) /2 ^[0,1) h- Obviously the probability of the set of paths starting from sq 
satisfying ip is 0. Since the CTMC is not stratified with respect to 99, Thm. I5TT1 cannot be 
applied directly: the product shall be constructed first. In the product CTMC, no states 
labelled with fa will be reached, thus giving the probability 0, as desired. 

Proof of Thm. \5.1\ We start with Eqn. (|5.ip . Let a\ and the other notation be as in the 
theorem. For s' £ S, define the event -^(s') := {c | = s' A \/t G [0,ai). a(t) \= 

consisting of paths which occupy state s' at time a\ and occupy /i-states during the 
time interval [0,ai). Obviously, {a \ a |= ip} C Us'U/! -^l]( s/ )0 Fix first a s as an initial 
distribution with a s (s) = 1 and s |= f\. By the law of total probability, we have: 

Prfo) = £ PrS^-O) • Pr?(^ I ^0) 

^Strictly speaking, this does not hold always because there may be paths that enter an (/2 A ~>fi)- 
state exactly at time ai; however, such paths are contained in a (generalization of) cylinder sets like 
Cyl(s, [01, oi], . . .), whose measure is 0. 
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The latter equality follows from the definition of ^^rjV). By the Markov property of CTMCs: 

= £ vr^^aO^O-Pr^^eG!) 

= J2^ Chfl] (s,a 1 )(s')-l s ^ fl -Pr c sl (i P Qa 1 ) 

s'&S 

where l s '\=fi is 1 if s ' \ = fi an d otherwise. Note that Pr^(yj) = ^2 s( zs a ( s )^ lc< s( l P) = 
E s ^/ a «( s ) Pr s (v). thus E q n - (ED follows. 

We now jump to the proof of Eqn. ()5.3j) . This proof is more involved, but follows the 
same lines. Define the event Z^s') := {a | = s' A Vi E [0, bi).a(t) |= /i. ..-,}. Again, 

{<T | <T |= 93} C U s '|=/ ., -^]( s/ )' an d again, fix a s as an initial distribution with a s (s) = 1 
and s |= /i...j. We have: 

s '\=fj'...j 

= ir Cl ^ ] (s,h)(s')-Pr C s (v\W» 

where the latter equality follows from the definition of Now let a E ^j(s'), thus 

<t(6i) = s', and a(t) |= /i...j for all < t < b±. Let a 1 denote the suffix path defined by 
cr'(x) := a(x + 61). 

Now, a |= <f implies that at time 61, a has reached a state in a stratum from qy,. ■ ■ , qj, 
so a' satisfies ipj> Q b\. On the other hand, every path a £ ^(^O whose corresponding 
o"' satisfies iff Q b\ also satisfies p> (because C is stratified). Again, ~Pr^(<p | ^j(s')) = 
Prf,^-/ e fei), thus 

s 'Ny...J 

However, C needs not be stratified w. r. t. (ffQbi, so to simplify the subsequent calculations, 
we restratify it: C\p /,'.. .fc— 1] is stratified w. r.t. 61. Eqn. (|5.3p for general initial 
distribution a follows as in the case of Eqn. (|5.ip . 

The proof for Eqn. (|5.2p is similar to the proof for Eqn. (|5.3p . except that b\ has to be 
replaced by aj and j' by 1. 

For Eqn. (I5.4p . we can again make a similar proof. First assume that j' = k. In that 
case, the paths that have reached an /&-state at any time in the interval Ik-i = I\ are 
exactly the paths that satisfy <p. They have the same probability as the paths in C[fk] that 
are in an /^-state exactly at time b±. Therefore, 

PrfM= 7r c ^( S ,6i)( S ') = X;^ [A] («,6i)(« , )-ly(=A ( 5 - 5 ) 
s'Hk s'eS 

With the usual assumption fk...k-i = false, the theorem follows immediately. 

If j' < k, besides the paths mentioned above, other paths satisfy 93, namely paths that 
reach an /^-state during the interval I^-i \ h = [&i,»fc-i) (and avoid /^-states earlier). 
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These are the paths that satisfy (/i A ->f k ) Ui x ... Ui k _ 2 (f k -i A U Ik _ 1 \ Il f k . Their 
probability is, according to Eqn. (|5.3p . 

£ vr ch(/,.,- 1 A^. ) ] (S;6l)(s0 . i s , h/j „ fc _ iA ^ A • Pr^'—W 60 

Note that C[— i(y r x...fc— l A "'/fe)] = £[/&]• Adding this term to Eqn. (|5.5p produces the desired 
probability. 

We still have to prove Eqn. (|5.4|) for 61 = 00. In that case, all timing constraints are 
trivial ([aj,6j) = [0, 00)) and j' = k. Therefore, Pr^(c^) is just the probability to reach an 
/fc-state eventually, which is exactly lim& 1 _>. 00 tt c ^ (s, 6i)I/ fc Pr^^ fe (fk). □ 

5.2. Closed Intervals. In Thm. 15.11 we have considered formula ip with left-closed inter- 
vals. Now we discuss that a slight generalization of it can be used to handle closed intervals. 
Thus, below we assume that Ii = \ai,bj\. 

The proof of Thm. ET] can be extended easily to hold also for closed intervals. Clause [3] 
may lead to formulas containing degenerate intervals [0,0]: As b\ £ [ai,&i], often j 1 = 1 in 
this clause. (We have to assume, as an additional simplification of notation, Iq := 0.) As a 
consequence, <pj> Qb\ = f\ f7[ 0)0 ] h Ui 2 eb 1 ■ ■ ■ fk- 

Further, if the original ip already contained a degenerate interval, say a\ = b±, so 
I\ = { a i} 5 applying Clause Q] will also lead to a formula containing [0,0]. These situations 
can be handled by the following lemma: 

Lemma 5.3. Let ip = f\ Ui 1 $2 Ui 2 ■ ■ ■ fk be a CSL path formula. Let C = (S, R, L) be a 
stratified CTMC with respect to ip. Moreover, assume I\ = . . . = Ij-x = [0, 0] for 2 < j < k. 
Then, Pr c s (<p) = Prf (/,■ I7j. . . . f k ) for all s G S. 

Proof. Assume a path a satisfies (p. The degenerate intervals force t\ = ti = . . . = tj-i = 0, 
thus no conditions relating to f±, . . . , fj-i need to be checked. □ 




/l)/3 /2 



Figure 5: A CTMC with Pr, (/i 17 [0>1] / 2 17 [1>3] / 3 ) ± Pr so (/i C/ [0 ,i) h U M h)- 

Example 5.4. Consider the CTMC in Fig.[5]and the path formula <p = fi ^[0,1] h ^[1,2] /3- 
Then, Pv So (p>) is the probability to stay in so for at least one time unit {tp(0, E(sq) • 1) 
in the notation of Section 16.31 below), since we can choose t± = ti = 1 if ctt[0] > 1. 
Applying Clause Oof Thm. EH we get j = 2, f = 1 and Pr So (p>) = TT C ^ fl -^(s , 1) • l fl 2 • 
Prf^ 1 ' 2l (/i U m h U m f 3 ) = 7r c ( S0j l)-I-Prf. ) (/ 2 C/ [0) i] /a) = (e~ 2 , 0) • (1, 0) T = e" 2 , the 
correct value. (In [22], we defined f slightly differently, producing j' = 2 and consequently 
Pr so (<^) = 0. Our earlier definition worked only for left-closed intervals.) 
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Below we apply the theorem to two formulas, and thereby get the well-known result [5] 
for the case of binary until for the case k = 2. As above, C is stratified and <p = fi Ui t 
Si U h ... f k . 

(1) Reachability probability. Assume that I\ = ... = Ik-i = [0,6]. Then, it holds Pr s ((^) = 
Pr s (O[ 0j fe]/fc)) which is the probability to reach an /^-state within time b. 

(2) Interval reachability. Assume that I± = . . . = Ik-i = [a, b] with a < b. Then, it holds 
Pr s ((p) = Yl s '\=fi ^ C ^^ (s, a)(s') ■ Pr s '(0[o i f ) _ a ]/fc), which is the interval reachability 
probability of staying in /i-states until time a and then moving to an /^.-state before 
time b has passed. 

5.3. Other Intervals. First, the following lemma states properties of the probabilities for 
binary until with different interval types: 

Lemma 5.5 (Closure of Intervals for Binary Until). Let s 6 S. Assume given two nonempty 
intervals I, J such that inf / = inf J and sup / = sup J. Then, it holds: 

(1) //0e/«0eJ, then s \=V< P (<5> Uj *) iff s \= V< p {$ Uj for < G {<,<,>,>}. 

(2) Otherwise, assume w. I. o. g. G / and J, and assume < p < 1. Then, s \= 
P>p($ Ui A $ iff s |= Uj 9), for > G {>,>}• Similarly, s |= V< p {<5> E/j 
*)V-$ iffs\=V< p (^Uj^), for <G {<,<}. 

The lemma follows immediately from the definition of the measure of cylinder set. To see 
why we have to treat the case inf 1 = separately (not distinguished in [5] ) , assume that 
$ = V<o.i(f2 ^(o,il /i) an d consider the CTMC depicted in Fig. [5j obviously we have 
so |= as so y= f2- However, so ty= V<o.i(f2 ^[0,1] Si) as s o satisfies f\ directly. The formula 
$ is equivalent to V< .i{f2 ^[0,1] Si) v ~'/2- 

For until formulas with arbitrary multiplicity, we have discussed the case that all of the 
intervals are left-closed or closed. Other cases can be handled in a way similar to Lemma f5.51 
However, to avoid too many technicalities, we skip these details. 

6. Model Checking Algorithm 

Let C = (S, R, L) be a CTMC, s G S, and $ be a CSL formula. The model checking 
problem is to check whether s |= <£. In the following two sections, we discuss that the 
model checking problem is decidable and provide an efficient algorithm for approximate 
computation of Pi s (ijj). 

6.1. Model Checking CSL is Decidable. The standard algorithm to solve CTL-like 
model checking problems recursively computes the sets of states satisfying ^, denoted 
by Sat(fy), for all state subformulas ^ of <£. For CSL, the cases where \& is an atomic 
proposition, a negation or a conjunction are given by: Sat(a) = {s G S \ a G L(s)}, 
Sat^x) = S\Sat($i) and Sat(^! A * 2 ) = n Sat($ 2 ). 

The case that ^ is the probabilistic operator is the challenging part. Let ^ = "P<ip(c/j) 
with ip = ^1 Ui 1 $2 Ui 2 . . . By the semantics, checking \t is equivalent to checking 
whether Pr s (<^) meets the bound < p, i.e., whether ~Pr s ((p) < p. Assume that the sets 
Sat(*5fi) have been calculated recursively. We replace V&i, . . . , ^ by fresh (pairwise different) 
atomic propositions fx, ■ ■ ■ ,Sk and extend the label of state s by fi if s G Sat(^i). The so 
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obtained path formula is tp := f\ Ui 1 fi Ui 2 ■ ■ ■ fk, and obviously we have Pi s (ip) = Pi s (tp). 
The steps needed to characterize Pr s (ip) are: 

(i) Construct the formula automaton B^. 

(ii) Build the product C x B^, which by Thm. |4"31 is a stratified CTMC w. r. t. ip. 

(iii) Apply Thm. [57X1 repeatedly to compute Pr <,(?/>)■ 

Thus, the decidability for the probabilistic formula reduces to checking whether Pi s (ip) < p 
holds true in the product CTMC. After applying Thm. 15.11 a finite number of times, we 
see that Pr s (ip) reduces to a product of transient probabilities. We can now follow the 
argumentation in [2]: Although the calculations differ slightly, Pi s (ijj) still is a finite sum 
X}fc% e<5 * (with algebraic i]k and 5k)- For such an expression, [2] proved that it can be 
decided whether it is < p, for p £ Q. Thus, we still have: 

Theorem 6.1 ([2], Thm. 1). Model checking CSL is decidable. 

6.2. Usefulness of Stratification. Our notion of stratified CTMCs solves a semantical 
problem in [2], which we recently pointed out in [13]. Very briefly, Aziz et al. [2] gave an 
algorithm that did not use the ti (in the semantics of until formulas) explicitly, which led 
to incorrect results for non-stratified CTMCs. 

Consider the CTMC depicted in Fig. [Hand the formula tp = f\ £/[o,i) /2 ^[o,i) /3 m 
Example 15.21 For this example, the algorithm in [2] calculates the probability that a path 
satisfies, a. o., the conditions: it stays in /i V/2-states during time [0, 1), thus giving a wrong 
result. This problem does not occur provided that the CTMC is stratified. 

6.3. Efficient Algorithm for Approximating Pr s {ip). We first explain how to combine 
steps (i) and (ii) mentioned above, without having to construct the full automaton B^. Most 
parts of the construction of C x B v depend on C only and do not require much information 
about B v . For example, for the state space, it is enough to generate k copies of every state 
in C, which requires time 0(\S\k). When constructing the transitions according to Clause[2] 
of Def. 14. 3| one has to check q' = 5(qi,L(s') n {fi, . . . but even this can be done 
without actually constructing B v by using the definition of 5 (Def. 14. ip directly. Therefore, 
the overall time complexity to find all transitions of C x B v is |R| times the number of copies 
that its source state may have, i.e., 0(|R|A;), which is also the maximal total number of 
transitions. 

The usual numerical algorithm to compute the matrix exponential e^* is based on 
uniformization |20j . This algorithm executes most calculations on the uniformized DTMC. 
For a CTMC, we say that A is a uniformization rate if A > max 5e g(i?(s) — R(s, s)). 

Definition 6.2. Let C = (5, R, L) be a CTMC. The uniformized DTMC of C with respect 
to the uniformization rate A is uni(C) = (S,P,L) where P(s,s') = R(s,s')/A if s 7^ s' and 
P(s,s) = l-P(s,S\{s}). 

Let P denote the transition matrix of the uniformized DTMC uni(C), thus it holds that 
P = / + Q/A where / denotes the identity matrix. For t > 0, then: 

vr(a, t) = n(a, 0) e ( p " 7 ) A * = vr(a, 0)e~ Ai £ ^Lp* = ^ \ t )v(i) (6.1) 

i=0 % ' i=0 
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In this formula, ip(i,Xt) = e~ xt ■ *-*f- denotes the i-th Poisson probability with parame- 
ter Xt, i.e., the probability to see precisely % transitions within time t. The vector v(i) is 
the transient probability of uni(C) after i transitions, i.e., v(i) = 7r(a,0)P l . The infinite 
sum is approximated, by picking 0(V~Xt) terms with large ifj(i,Xt), using the Fox-Glynn 
algorithm [9JQ3]. To find the v(i) for Eqn. (|6.ip . one requires O(Xt) matrix-vector multi- 
plications [5]. The following lemma states the complexity of our algorithm: 

Lemma 6.3 (Complexity). Let |R| denote the number of transitions of C and A S M>o the 
uniformization rate satisfying A = max sg s(.E(s) — R(s, s)). For each formula <p = fi Uj 1 
$2 Ui 2 . . . fk, the probability Prf*^^) can be approximated: 

• in time in 0(\R\k ■ Xb) if b = sup//^ is finite, 

• in time in 0(\H\k ■ Xb + (|<S|A;) 3 ) if sup I^-i is infinite, where \S\ is the number of states 
in C and b = max ({inf Ik-i} U {sup/j|l < i < k} \ {oo}). 

The space complexity is in 0(\H\k). 

Proof. Recall that the formula automaton is deterministic, and the size of the product 
automaton is 0(|R|A;) which is both linear in the size of the CTMC and the formula. This 
proves the space complexity. 

For the time complexity assume first b < oo with b = sup/fc„i. Applying Thm. 15. 1\ 
the probability Pr^^ (ip) can be expressed as a sequence of transient probability analyses, 
which can be efficiently approximated by a sequence of uniformization analyses. The com- 
plexity of these analyses is linear in the size of the product automaton, and also linear in 
Xb. 

For the second case sup/fc„i = oo, by Thm. 15. 1\ a sequence of transient probability 
analyses is followed by one steady-state analysis, which can be done with Gaussian elimi- 
nation for the equation systems ir ■ Q' = and ^2 s€ g/ 7r(s) = 1, the complexity of which is 
0((\S\k) 3 ). Thus the complexity for this case follows. □ 

Thus, with the notion of stratified CTMC, we achieve polynomial complexity. Our al- 
gorithm therefore improves the work of [2] , where only multiple until formulas with suitable 
timing constraints can be checked polynomially. In the worst case, [2] has to decompose a 
CSL formula into 0((k — formulas with suitable timing, thus resulting in an overall 

time complexity of C(|R| • Xb(k - l) k ) or 0({\R\k • Xb + (|S"|A;) 3 ) • (k - l) fc_1 ), respectively. 

7. Related Work 

The logic CSL was first proposed in [I], in which the model checking problem is shown to 
be decidable. Our paper gives a practical solution: it shows that the relevant probabilities 
can be approximated efficiently. For the case of binary until path formula, Baier et al. [5] 
have presented an approximate algorithm for the model checking problem. Their method 
can be considered a special case of our approach. 

Baier et al. [3] defined a logic asCSL that uses so-called programs as path formulas, 
i. e. regular expressions over state formulas and actions. Programs can express multiple 
until formulas of the form <pi i/[o.&) ^2 £qo,6) "' ^[0,6) Vfc because asCSL cannot restrict 
the duration of individual program phases. The model checking algorithm translates the 
program to an automaton almost equal to the one in Fig. [2j Our work generalizes the 
method to multiple until formulas with multiple time bounds. 
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More recently, Donatelli et al. [8] have extended CSL such that path properties can be 
expressed via a deterministic timed automata (DTA) with a single clock. Chen et al. [7] 
take this approach further and consider DTA specifications with multiple clocks as well. 

In principle, one can translate a multiple until formula to a DTA with a single clock. Its 
basic structure would look similar to Fig. [2 but Donatelli's and Chen's DTAs also include 
all timing information and would have a size in 0(k 2 ) - an example construction with 
k = 4 is given in Appendix [A] To check whether a CTMC satisfies a DTA specification, 
they build the product of the two, apply the region construction, and then solve a system of 
integral equations. Chen's method, applied directly to our specifications, would amount to a 
complexity in 0(/c 4 |S'|Ac+A; 9 |5'| 3 ), where c is the largest difference between time constraints 
(roughly comparable to b in Lem. 16. 3p . Note that our algorithm has only a complexity in 
0(\TL\k-\b) if6 = sup/ fc _i < oo or C(|R|fc • Xb + (|5|A:) 3 ) otherwise. 

8. Conclusion 

In this paper we have proposed an effective approximation algorithm for CSL with a multiple 
until operator. We believe that it is the centerpiece of a broadly applicable full CSL model 
checker. 

The technique we have developed in this paper can also be applied to a subclass of 
PCTL* formulas. Let ip = fx Uj 1 fi C/j 2 ■ ■ ■ fk be a CSL path formula. As we have seen in 
the paper, in case of I\ = . . . = Ik-i = [0, oo), our multiple until formula f\ Ui fiVi . . . fk 
corresponds to the LTL formula f\ U (fi U (. . . (fk-i U fk) ■ ■ •))• I n general, tp is similar to 
a step-bounded LTL formula <p = fx f/^jj] fi ^J\i 2 .j 2 ] ■ ■ ■ fk with ii, j±, . . . integers specifying 
the step bounds. Such step-bounded until LTL formulas can be first transformed into nested 
next-state formulas, for example we have: f\ U^ 2 .3] fi = fx A X(fi A X(f 2 V (fi A X(f2))))- 
The approach we have established in this paper can be adapted slightly to handle this kind 
of formulas in complexity linear in jk-i (assuming jk-i < oo). 

We conclude the paper by noting the connection of our DFA-based approach with the 
classical Biichi- automaton-based LTL model checking algorithm by Vardi and Wolper |21j . 
The LTL formula ip is first transformed into a Biichi automaton - of exponential size in the 
worst case - accepting exactly the words satisfying ip. Then, model checking LTL can be 
reduced to automata-theoretic questions in the product. Instead of Biichi automata accept- 
ing infinite runs, we only need DFAs, which is due to the simple form of the multiple until 
formula: it does not encompass the full expressivity of LTL. This simplification, moreover, 
allows us to get a DFA whose number of states is only linear in the length of the CSL 
formula, and the size of the product automaton is then linear in both the size of the CTMC 
and the length of the CSL formula. 
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Figure 6: A deterministic CSL -timed automaton for fi J7[ ai ,6i) H U[a 2 ,b 2 ) f'3 ^[a 3 ,ft 3 ) f&- 

Appendix A. Translating Fig. |to a DTA for CSL ta 

As mentioned in Section Donatelli et al. [8] have extended CSL such that path properties 
can be expressed via a timed automaton. In Fig. [6l we include a DTA corresponding to the 
formula f\ £7[ oli6l ) f 2 f7[ a2 ,& 2 ) h ^[03,63) fa with < a x < a 2 < a 3 < b x < b 2 < b 3 . Dashed 
lines correspond to transition edges; solid lines to boundary edges. The automaton has a 
single clock x. The state label qij indicates that time has passed and that the current 
time is in the interval /. The guard x < 63 is needed in states without boundary edge to 
ensure that 54 is not entered too late. 

The automaton illustrates that CSL TA may need a DTA with 0(k 2 ) states, where k is 
the number of phases in the multiple until-formula. 
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